Techniques to manage training or trained models for deep learning applications

ABSTRACT

Examples include techniques to manage training or trained models for deep learning applications. Examples include routing commands to configure a training model to be implemented by a training module or configure a trained model to be implemented by an inference module. The commands routed via out-of-band (OOB) link while training data for the training models or input data for the trained models are routed via inband links.

CLAIM FOR PRIORITY

This application is a continuation of and claims priority to co-pendingU.S. patent application Ser. No. 17/208,861, filed on Mar. 22, 2021, andtitled “Techniques to Manage Training or Trained Models for DeepLearning Applications”, which is a continuation of U.S. patentapplication Ser. No. 17/025,643, filed on Sep. 18, 2020, and titled“Techniques to Manage Training or Trained Models for Deep LearningApplications”, which is a continuation of U.S. patent application Ser.No. 15/859,472, filed on Dec. 30, 2017, and titled “Techniques to ManageTraining or Trained Models for Deep Learning Applications”, which isincorporated by reference in its entirety.

TECHNICAL FIELD

Examples described herein are generally related to a platform forartificial intelligence (AI) training appliances or AI inferenceappliances.

BACKGROUND

Deep learning applications may employ the use of artificial intelligence(AI) or machine learning. The use of AI may include using trainingresources to build or construct trained models based on processing largeamounts of training data. For example, processing millions of images ofdifferent faces to build facial recognition models to identify a groupof people or even specific individuals. Once a model is constructed ortrained, the trained model may be implemented by inferencing resources.These inferencing resources may process data (e.g., various types ofimages) as data inputs to trained models to arrive at results (e.g.,facial recognition).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example first system.

FIG. 2 illustrates example second system.

FIG. 3 illustrates an example process.

FIG. 4 illustrates an example block diagram for a first apparatus.

FIG. 5 illustrates an example of a first logic flow.

FIG. 6 illustrates an example of a first storage medium.

FIG. 7 illustrates an example block diagram for a second apparatus.

FIG. 8 illustrates an example of a second logic flow.

FIG. 9 illustrates an example of a second storage medium.

FIG. 10 illustrates an example third system.

DETAILED DESCRIPTION

As deep learning applications become more common and more complex, aneed for deep learning infrastructure having integrated AI capabilitiesfor training and inferencing resources is growing for entities deployingor using these deep learning applications. These integrated AIcapabilities may need to be deployed and used readily across a breadthof different scenarios ranging from edge, internet of things (IoT),cloud services for mobile devices or to data centers. Each scenario mayhave varying levels of sophistication.

Deep learning infrastructure having integrated AI capabilities fortraining and inferencing resources may need to be transparent, powerful,power-efficient and flexible for entities deploying or using deeplearning applications. Just as cloud infrastructure permits easyapplication of a certain amount of computational and power budgetflexibility among different workloads or needs at different times, asimilar need for possibly dynamically assigning deep learninginfrastructure may apply. This may mean not having to burn power anddelay deep learning results generated by training and/or inferencingresources due to under or overutilized processing resources.

A typical way to field specialized AI hardware for deep learninginfrastructure is to have bootable central processing units (CPUs) toact as hosts for training resources and/or inferencing resources (e.g.,specialized devices/accelerators). Even some types of training resourcessuch as a graphics processing unit (GPU) may need to be managed from ageneral-purpose CPU. CPUs acting as hosts for specialized AI hardware ormanaging a GPU used for training may add additional operating overhead.Also, these types of deep learning infrastructures may become hard tomodify or change as power and cost may dictate how resources included inthese types of deep learning infrastructures are configured. Power andcost dictating resource configurations may result in a focus more onattaching power efficient CPUs to training and inferencing resources, orover-designing platforms with powerful CPUs for managing deep learninginfrastructure that may result in costly and/or inefficient platforms.It is with respect to these challenges that the examples describedherein are needed.

FIG. 1 illustrates an example system 100. In some examples, as shown inFIG. 1, system 100 includes a drawer 120 that includes training modules101-1 to 101-n and inference modules 102-2 to 102-n, where “n”represents any whole positive integer greater than 1. For theseexamples, drawer 120 also includes an out-of-band (OOB) training modelmanagement logic 108 coupled with training modules 101-1 to 101-n viarespective OOB links 112-1 to 112-n and an OOB trained model managementlogic 109 coupled with inference modules 102-1 to 102-n via respectiveOOB links 113-1 to 113-n. Drawer 120 also includes an inband trainingdata interface and switch 110 coupled with training modules 101-1 to101-n via respective inband links 106-1 to 106-n and inband input datainterface and switch 111 coupled with inference modules 102-1 to 102-nvia respective inband links 107-1 to 107-n. Also, as shown in FIG. 1, anAI training to inference switch 105 may be coupled with training modules101-1 to 101-n via respective switch links 115-1 to 115-n and coupledwith inference modules 102-1 to 102-n via respective switch links 116-1to 116-n.

According to some examples, drawer 120 may represent a platformcomprised of a single circuit board or multiple physically coupledcircuit boards that include both training and inferencing resources.Drawer 120 may be inserted as a drawer for a larger computing platformor system with several other drawers that may also include training andinferencing resources (e.g., configured as a rack). The larger computingplatform or system may also include other computing resources inseparate drawers such as drawers including CPUs or storage resources.Drawer 120 may be configured in a similar manner as a sled or bladeinserted in a larger computing platform such as rack.

In some examples, OOB links 112-1 to 112-n or OOB links 113-1 to 113-nmay be configured as sideband communication links that may transmitrequests or management data to training modules 101-1 to 101-n orinference modules 102-1 to 102-n to manage or configure these modules(described more below). For these examples, OOB links 112-1 to 112-n orOOB links 113-1 to 113-n may operate using communication protocols suchas, but not limited to Ethernet communication protocols, such asdescribed in one or more Ethernet standards promulgated by the Instituteof Electrical and Electronics Engineers (IEEE). For example, one suchEthernet standard promulgated by IEEE may include IEEE 802.3-2012,Carrier sense Multiple access with Collision Detection (CSMA/CD) AccessMethod and Physical Layer Specifications, Published in December 2012(hereinafter “IEEE 802.3 specification”). Also, OOB links 112-1 to 112-nor OOB links 113-1 to 113-n may transmit requests or management data torespective training modules 101-1 to 101-n or inference modules 102-1 to102-n in communication links that are separate from inband links 106-1to 106-n or 107-1 to 107-n used to transmit training or input data torespective training modules 101-1 to 101-n or inference modules 102-1 to102-n. Also, since training modules 101-1 to 101-n or inference modules102-1 to 102-n may receive request or management data relatively rarelycompared to these modules receiving training or input data, OOB links112-1 to 112-n or OOB links 113-1 to 113-n may be arranged to supportlower data bandwidths compared to inband links 106-1 to 106-n or 107-1to 107-n.

According to some examples, inband links 106-1 to 106-n or 107-1 to107-n may include high speed and/or high bandwidth communication linksto route training or input data to training modules 101-1 to 101-n orinference modules 102-1 to 102-n. For these examples, inband links 106-1to 106-n or 107-1 to 107-n may include the use of a high speedserializer and deserializer (SerDes) to enable data bandwidths rangingfrom several gigabits-per-second to over a terabit-per-second. Theserelatively high bandwidths, for example, may allow large amounts oftraining data received by inband training data interface and switch 110to be routed to destination training modules from among training modules101-1 to 101-n for use in training models 101-1A/B to 101-nA/B. Therelatively high bandwidths, for example, may also allow large amounts ofinput data to be received by inband input data interface and switch 111and routed to destination inference modules from among inference modules102-1 to 102-n for use in trained models 102-1A/B to 102-nA/B.

In some examples, as shown in FIG. 1, training modules 101-1 to 101-nmay couple with AI training to inference switch 105 via respectiveswitch links 115-1 to 115-n. Also, as shown in FIG. 1, inference modules102-1 to 102-n may couple with AI training to inference switch 105 viarespective switch links 116-1 to 116-n. As described more below, switchlinks 115-1 to 115-n and 116-1 to 116-n may be used to route trainedmodels generated by training modules 101-1 to 101-n to at least one ofinterference modules 102-1 to 102-n. For these examples, AI training toinference switch 105 may be configured as a software or hardware type ofswitch to route trained models to destination inference modules fromamong inference modules 102-1 to 102-n. According to some examples,training to inference switch 105 may be configured to route trainedmodels to destination inference modules using one or more types ofcommunication protocols, such as, but not limited to, Ethernet,Infiniband, Fibre Channel, or transmission control protocol (TCP).Switch links 115-1 to 115-n and 116-1 to 116-n may also have relativelylower data bandwidths compared to data bandwidths for inband links 106-1to 106-n or 107-1 to 107-n.

According to some examples, OOB training module management logic 108 mayinclude logic and/or features to receive management data and perform OOBmanagement of training modules 101-1 to 101-n. For these examples, themanagement data may be received via a generic interface (not shown)coupled with OOB training module management logic 108. The genericinterface may provide a means to receive requests to configure trainingmodels being implemented by training modules 101-1 to 101-n or tocontrol/manage various functions of these training modules via commandssent over OOB links 112-1 to 112-n. For example, commands such as, butnot limited to, starting or stopping implementation of training models,commands to change one or more priorities associated with implementingtraining models, or commands to set/adjust quality of service (QoS)priorities and service level agreements (SLAs) for training models ortraining modules (e.g., to specify bandwidths between producers andconsumers for these training resources).

In some examples, similar to OOB training module management logic 108,OOB trained model management logic 109 may include logic and/or featuresto receive management data and perform OOB management of inferencemodules 102-1 to 102-n and/or trained models implemented by theseinference modules. For these examples, the management data may bereceived via a generic interface (not shown) coupled with OOB trainedmodule management logic 109. The generic interface may provide a meansto receive requests to configure inference modules 102-1 to 102-n or tocontrol/manage various functions of these inference modules via commandssent over OOB links 113-1 to 113-n. For example, commands such as, butnot limited to, starting or stopping implementation of trained models,commands to change one or more priorities associated with implementingtrained models, or commands to set/adjust QoS priorities and SLAs fortrained models or inference modules.

According to some examples, use of OOB training module management logic108 and OOB trained model management logic 109 coupled with separategeneric interfaces to receive management data and perform OOB managementvia separate OOB links, may enable managing/configuring training modules101-1 to 101-n or managing/configuring inference modules 102-1 to 102-nto be handled such that the underlying infrastructure included in drawer120 may be software-defined, transparent and flexible. Software-defined,transparent and flexible may be desirable traits for a deep learninginfrastructure having integrated training and inference capabilitiessuch as shown in FIG. 1 for drawer 120. Also, the separate managementlogic and OOB links for managing training and inference modules may addrobustness and resiliency.

In some examples, as shown in FIG. 1, OOB training module managementlogic 108 may be coupled with model generation logic 103-1 to 103-n atrespective training modules 101-1 to 101-n via OOB links 112-1 to 112-n.For these examples, responsive to management/control by OOB trainingmodule management logic 108, model generation logic 103-1 to 103-n maybe arranged to adapt and optimize trained models generated as a resultof implementing training models 101-1A/B or 101-nA/B. These trainedmodels may be translated into bit-ware (e.g., executable by a fieldprogrammable gate array (FPGA)) by model generation logic 103-1 to 103-nand then forwarded to AI training to interference switch 105 viarespective switch links 115-1 to 115-n. AI training to interferenceswitch 105 may then forward trained models to one or more destinationinference modules of inference modules 102-1 to 102-n for loading orprogramming of trained models to the one or more destination inferencemodules.

In some examples, as shown in FIG. 1, OOB trained module managementlogic 109 may be coupled with request and registration logic 114-1 to114-n at respective inference modules 102-1 to 102-n via OOB links 113-1to 113-n. For these examples, responsive to management/control by OOBtrained module management logic 109, registration logic 114-1 to 114-nmay been arranged to set up trained models either forwarded to inferencemodules 102-1 to 102-n from AI training to inference switch 105 oruploaded and received from OOB trained module management logic 109. Forexample, trained model 102-1A at inference module 102-1 may haveoriginated from a trained model generated by training module 101-1 thatwas translated to bit-ware by model generation logic 103-1 and routedthrough AI training to inference switch 105 to registration logic 114-1for set up at inference module 102-1. Meanwhile, trained model 102-1Bmay have been uploaded by OOB trained module management logic 109 andthen sent to registration logic 114-1 for set up at inference module102-1.

According to some examples, training modules 101-1 to 101-n may includeprocessing elements such as application specific integrated circuits(ASICs) or other types of specialized processing elements such asgraphics processing units (GPUs) or neural network processors (NNPs).Also, inference modules 102-1 to 102-n may include programmableprocessing elements such as FPGAs. For these examples, model generationlogic 103-1 to 103-n may be configured by OOB training module managementlogic 108 for effective and efficient uses of different categories ofFPGAs that may be added or removed from inference modules 102-1 to102-n. This may allow drawer 120 to be upgraded with newer or differentFPGAs and/or have a changing variety of FPGAs to address a range ofpossibly changing categories or problems being handled by particular AIoperations associated with one or more deep learning applications. Forexample, deep learning applications such as facial or voice recognitionto identify a person or a group of people for security and/or socialmedia purposes.

FIG. 2 illustrates example system 200. In some examples, as shown inFIG. 2, system 200 may include a drawer 220 and a drawer 230 coupledthrough an AI training to inference switch 205. For these examples,system 200 includes similar elements as shown in FIG. 1 for system 100,but system 200 has training resources and inference resources onseparate drawers. Drawers 220 and 230 may be inserted as drawers in alarger computing platform or system. Like drawer 120, drawers 220 and230 may also be configured as a sled or blade in a larger computingplatform such as a rack. Management or control of training modules 201-1to 201-n or of inference modules 202-1 to 202-n may be done in a similaror same manner as mentioned previously for training modules 101-1 to101-n or of inference modules 102-1 to 102-n.

In some examples, as shown in FIG. 2, AI training inference switch 205may be physically separate from drawer 220 or 230. For these examples,switch links 215-1 to 215-n from drawer 220 and switch links 216-1 to216-n to drawer 230 may be routed through a backplane of a largercomputing platform or system that includes drawers 220 and 230.

According to some examples, AI training inference switch 205 may be apart of either drawer 220 or drawer 230. For these other examples, thedrawer not including AI training inference switch 205 may have itsswitch links routed through a backplane of a larger computing platformthat includes drawer 220 and drawer 230.

In some examples, AI training inference switch 205 may be located on adrawer, sled or blade that is separate from drawer 220 or drawer 230.For these examples, AI training inference switch 205 may server as aswitch for just drawer 220 or drawer 230 or may serve as a switch formultiple drawers for training modules routing trained models to one ormore drawers for inference modules. Switch links 215-1 to 215-n orswitch links 216-1 to 216-n may be routed through a backplane of alarger computing platform that includes drawers 220, 230 and the drawerincluding AI training inference switch 205.

According to some examples, rather than coupling to a single drawerthrough AI training to inference switch 205 as shown in FIG. 2, drawer220 may couple to multiple drawers configured similar to drawer 230. Forthese examples, trained models generated by training modules 201-1 to201-n may be routed through AI training to inference switch 205 toinference modules located on one or more drawers having switch linkscoupled with AI training to interference switch 205.

FIG. 3 illustrates an example process 300. In some examples, process 300may be an example process of how training and inference modules may bemanaged or configured to support AI operations. For these examples,elements of system 100 as shown in FIG. 1 may be related to process 300.These elements of system 100 may include, but are not limited to, OOBtraining model management logic 108, inband training data interface andswitch 110, training module 101-1, AI training to inference switch 105,inference module 102-1, OOB trained model management logic 109 or inbandinput data interface and switch 111.

Beginning at process 3.1 (Receive Management Data), OOB training modelmanagement logic 108 may receive management data. In some examples, themanagement data may include a request to configure or manage aparticular training model being implemented by training module 101-1 fora deep learning application. For example, the management data mayrequest to configure or manage a training model such as training model101-1B that may be used for such deep learning applications as facial,image or voice recognition for individuals or groups of individuals. Themanagement data may also indicate an operation that is to be performedas part of the request. The operation may include, but is not limitedto, starting or stopping the training model or reconfiguring one or moreparameters of the training model (adding/subtracting inputs or changingone or more training algorithms).

Moving to process 3.2 (Config./Manage), OOB training model managementlogic 108 may configure or manage the training model indicated in thereceived management data. For these examples, OOB training modelmanagement logic 108 may utilize a training lookup table that indicateswhat training module is implementing a given training model to determinewhich training module is implementing the particular training module.For example, the management data may indicate a request to configure ormanage training model 101-1B and OOB training model management logic 108may utilize the training lookup table to match training model 101-1B totraining module 101-1. In other examples, the management data mayindicate that training model 101-1B is a new training model. For theseother examples, OOB training model management logic 108 may determinewhat training module can implement the new training model and thenupdate the training lookup table based on that determination. Thisdetermination may include load-balancing training models to beimplemented between one or more of training modules 101-1 to 101-n orpossibly removing one or more training models if all training modulesare at capacity or overloaded. Removing one or more training modelsimplemented at a training module may enable training model 101-B toreplace the removed training model at that training module. For theseother examples, the update to the training lookup table, for example,may indicate that training model 101-1B is to be implemented by trainingmodule 101-1.

According to some examples, OOB training model management logic 108 mayconfigure or manage training model 101-1B implemented by training module101-1 by issuing one or more training commands. The training commandsmay be routed via OOB link 112-1 and may include or may be based on thereceived management data. An example format for training commands routedvia OOB link 112-1 may be “ManageTraining(Model_ID, Op)”, where Model_IDindicates training model 101-1B and Op indicates the desired operationto be configured or managed. Model_ID may a unique identifier that mayhave been agreed upon previously. For example, a previously agreed uponuniversal unique identifier (UUID).

Moving to process 3.3 (Generate/Modify Training Model), training module101-1 may generate training model 101-1B if new or modify training model101-B if already being implemented by training module 101-1 for a deeplearning application (e.g., facial recognition of a particularindividual or individuals) for which training model 101-1B is to betrained. Generating or modifying training model 101-B may be based onthe operation indicated in the management data received by OOB trainingmodel management logic 108.

Moving to process 3.4 (Receive Training Data), inband training datainterface and switch 110 may receive training data to be used intraining model 101-1B implemented by training module 101-1 for the deeplearning application for which training model 101-1B is to be trained.In some examples, the training data may be in a format of“Train(Model_ID, Data)”, where Model_ID identifies training model 101-1Band Data includes the training data to be used in training model 101-1B.

Moving to process 3.5 (Route Training Data), inband training datainterface and switch 110 may utilize the training lookup table todetermine that training module 101-1 is to implement training model101-1B for the deep learning application and then route the trainingdata to training module 101-1 based on that determination.

Moving to process 3.6 (Generate Trained Model), model generation logic103-1 at training module 101-1 may generate a trained model based onreceived training data processed by training module 101-1 whileimplementing training model 101-1B for the deep learning application. Insome examples, the generated trained model may be translated by modelgeneration logic 103-1 to bit-ware and then identified as trained model102-1B.

Moving to process 3.7 (Send Trained Model), model generation logic 103-1at training module 101-1 may cause trained model 102-1B to be sent to AItraining to inference switch 105. In some examples, trained model 102-1Bmay be sent via switch link 115-1.

Moving to process 3.8 (Forward Trained Model(s)), AI training toinference switch 105 may forward trained model 102-1B to inferencemodule 102-1. In some examples, request and registration logic 114-1 mayreceive trained model 102-1B and may send an indication to OOB trainedmodel management logic 109 via OOB link 113-n that trained model 102-1Bis ready to be set up at inference module 102-1.

Moving to process 3.9 (Set Up Trained Model), request and registrationlogic 114-1 may receive an indication from OOB trained model managementlogic 109 approving at least an initial set up of trained model 102-1Bat inference module 102-1. In some examples, approval may be based onload considerations for inference module 102-1. For these examples, OOBtrained model management logic 109 may update an inference lookup tablethat indicates trained model 102-1B is being implemented by inferencemodule 102-1 for the deep learning application.

Moving to process 3.10 (Receive Management Data), OOB trained modelmanagement logic 109 may receive management data. According to someexamples, the management data may include a request to configure ormanage trained model 102-1B for the deep learning application. Themanagement data may also indicate an operation that is to be performed.The operation may include, but is not limited to, starting or stoppingtrained model 102-1B or reconfiguring one or more parameters of trainedmodel 102-1B.

According to some examples, OOB trained model management logic 109 mayconfigure or manage trained model 102-1B implemented by inference module102-1 by issuing one or more trained commands. The trained commands maybe routed via OOB link 113-1 and may include the received managementdata. An example format for the trained commands routed via OOB link113-1 may be “ManageInference(Model_ID, Op)”, where Model_ID indicatestrained model 102-1B and Op indicates the desired operation.

Moving to process 3.11 (Config./Manage), OOB trained model managementlogic 109 may configure or manage trained model 102-1B according to thereceived management data. In some examples, OOB trained model managementlogic 109 may refer to the inference lookup table to determine thatinference module 102-1 is implementing trained model 102-1B for the deeplearning application.

Moving to process 3.12 (Receive Input Data), inband input data interfaceand switch 111 may receive input data to be used in trained model101-1AB implemented by inference module 102-1. In some examples, thetraining data may be in a format of “Inference(Model_ID, Payload)”,where Model_ID identifies trained model 102-1B and Payload includes theinput data to be used in trained model 102-1B.

Moving to process 3.13 (Route Input Data), inband input data interfaceand switch 111 may utilize the inference lookup table to determine thatinference module 102-1 is implementing trained model 102-1B and thenroute the training data to inference module 102-1 based on thatdetermination.

Moving to process 3.14 (Apply Input Data to Trained Model), the inputdata may be applied to trained model 102-1A implemented by inferencemodule 102-1 for the deep learning application. In some examples, theapplied input data may cause inference module 102-1 to return a resultfor the deep learning application.

FIG. 4 illustrates an example block diagram for apparatus 400. Althoughapparatus 400 shown in FIG. 4 has a limited number of elements in acertain topology, it may be appreciated that the apparatus 400 mayinclude more or less elements in alternate topologies as desired for agiven implementation.

According to some examples, apparatus 400 may be included or embodied asOOB training model management logic 108 or 208 as shown in FIG. 1 or 2and may be supported by circuitry 420 maintained at OOB training modelmanagement logic 108 or 208. Circuitry 420 may include a processorcircuit arranged to execute one or more software or firmware implementedmodules, components or logic 422-a (module, component or logic may beused interchangeably in this context). In other examples, circuitry 420may be arranged to implement modules, components or logic 422-a that maybe wholly or at least partially implemented in hardware (module,component or logic may also be used interchangeably in this context). Itis worthy to note that “a” and “b” and “c” and similar designators asused herein are intended to be variables representing any positiveinteger. Thus, for example, if an implementation sets a value for a=3,then a complete set of modules, components or logic 422-a may includelogic 422-1, 422-2 or 422-3. The examples presented are not limited inthis context and the different variables used throughout may representthe same or different integer values. Also, “logic”, “module” or“component” may also include software/firmware stored incomputer-readable media, and although types of logic are shown in FIG. 4as discrete boxes, this does not limit these types of logic to storagein distinct computer-readable media (e.g., a separate memory, etc.).

According to some examples, as mentioned above, circuitry 420 mayinclude a processor circuit. Circuitry 420 may be generally arranged toexecute or implement one or more components, modules or logic 422-a. Insome examples circuitry 420 may be configured as an application specificintegrated circuit (ASIC) and at least some components, modules or logic422-a may be implemented as hardware elements of the ASIC. According tosome examples, circuitry 420 may be configured as a field programmablegate array (FPGA) and at least some components, modules or logic 422-amay be implemented as hardware elements of the FPGA.

In some examples, apparatus 400 may include a receive logic 422-1.Receive logic 422-1 may be executed or implemented by circuitry 420 toreceive, via a link, management data to configure a training model to beimplemented by a training module for a deep learning application. Forthese examples, the management data may be included in management data410.

According to some examples, apparatus 400 may also include an identifylogic 422-2. Identify logic 422-2 may be executed or implemented bycircuitry 420 to identify the training module to implement the trainingmodel. For these examples, identify logic 422-2 may maintain mappinginformation 423-a (e.g., in a lookup table) that may indicate whichtraining models are being implemented by which training module fromamong a plurality of training modules. Identify logic 422-2 may usemapping information 423-a to identify the training module.

In some examples, apparatus 400 may also include a command logic 422-3.Command logic 422-3 may be executed or implemented by circuitry 420 tocause the training module to be configured responsive to the managementdata based on one or more commands routed to the training module via anOOB link. For these examples, the training module may receive trainingdata for the training model via an inband link and generate a trainedmodel based on using the training data to implement the training model.The one or more commands may be included in commands 430.

Various logic, modules or components of apparatus 400 may becommunicatively coupled to each other by various types of communicationsmedia to coordinate operations. The coordination may involve theuni-directional or bi-directional exchange of information. For instance,the logic, modules or components may communicate information in the formof signals communicated over the communications media. The informationcan be implemented as signals allocated to various signal lines. In suchallocations, each message is a signal. Further embodiments, however, mayalternatively employ data messages. Such data messages may be sentacross various connections. Example connections include parallelinterfaces, serial interfaces, and bus interfaces.

Included herein is a set of logic flows representative of examplemethodologies for performing novel aspects of the disclosedarchitecture. While, for purposes of simplicity of explanation, the oneor more methodologies shown herein are shown and described as a seriesof acts, those skilled in the art will understand and appreciate thatthe methodologies are not limited by the order of acts. Some acts may,in accordance therewith, occur in a different order and/or concurrentlywith other acts from that shown and described herein. For example, thoseskilled in the art will understand and appreciate that a methodologycould alternatively be represented as a series of interrelated states orevents, such as in a state diagram. Moreover, not all acts illustratedin a methodology may be required for a novel implementation.

A logic flow may be implemented in software, firmware, and/or hardware.In software and firmware embodiments, a logic flow may be implemented bycomputer executable instructions stored on at least one non-transitorycomputer readable medium or machine readable medium, such as an optical,magnetic or semiconductor storage. The embodiments are not limited inthis context.

FIG. 5 illustrates an example logic flow 500. Logic flow 500 may berepresentative of some or all of the operations executed by one or morelogic or devices described herein, such as logic included in apparatus400. More particularly, logic flow 500 may be implemented by at leastreceive logic 422-1, identify logic 422-2 or command logic 422-3.

According to some examples, logic flow 500 at block 502 may receivemanagement data to configure a training model to be implemented by atraining module for a deep learning application. For these examples,receive logic 422-1 may receive the management data.

In some examples, logic flow 500 at block 504 may identify the trainingmodule to implement the training module. For these examples, identifylogic 422-2 may identify the training module.

According to some examples, logic flow 500 at block 506 may cause thetraining module to be configured responsive to the management data basedon one or more commands routed to the training module via an OOB link,the training module to receive training data for the training model viaan inband link and generate a trained model based on using the trainingdata to implement the training model. For these examples, command logic422-3 may cause the training module to be configured responsive to themanagement data based on the one or more commands.

FIG. 6 illustrates an example storage medium 600. As shown in FIG. 6,the first storage medium includes a storage medium 600. The storagemedium 600 may comprise an article of manufacture. In some examples,storage medium 600 may include any non-transitory computer readablemedium or machine readable medium, such as an optical, magnetic orsemiconductor storage. Storage medium 600 may store various types ofcomputer executable instructions, such as instructions to implementlogic flow 500. Examples of a computer readable or machine readablestorage medium may include any tangible media capable of storingelectronic data, including volatile memory or non-volatile memory,removable or non-removable memory, erasable or non-erasable memory,writeable or re-writeable memory, and so forth. Examples of computerexecutable instructions may include any suitable type of code, such assource code, compiled code, interpreted code, executable code, staticcode, dynamic code, object-oriented code, visual code, and the like. Theexamples are not limited in this context.

FIG. 7 illustrates an example block diagram for apparatus 700. Althoughapparatus 700 shown in FIG. 7 has a limited number of elements in acertain topology, it may be appreciated that the apparatus 700 mayinclude more or less elements in alternate topologies as desired for agiven implementation.

According to some examples, apparatus 700 may be included or embodied asOOB training model management logic 108 or 208 as shown in FIG. 1 or 2and may be supported by circuitry 720 maintained at OOB trained modelmanagement logic 109 or 209. Circuitry 720 may include a processorcircuit arranged to execute one or more software or firmware implementedmodules, components or logic 722-a (module, component or logic may beused interchangeably in this context). In other examples, circuitry 720may be arranged to implement modules, components or logic 722-a that maybe wholly or at least partially implemented in hardware (module,component or logic may also be used interchangeably in this context). Itis worthy to note that “a” and “b” and “c” and similar designators asused herein are intended to be variables representing any positiveinteger. Thus, for example, if an implementation sets a value for a=3,then a complete set of modules, components or logic 722-a may includelogic 722-1, 722-2 or 722-3. The examples presented are not limited inthis context and the different variables used throughout may representthe same or different integer values. Also, “logic”, “module” or“component” may also include software/firmware stored incomputer-readable media, and although types of logic are shown in FIG. 7as discrete boxes, this does not limit these types of logic to storagein distinct computer-readable media (e.g., a separate memory, etc.).

According to some examples, as mentioned above, circuitry 720 mayinclude a processor circuit. Circuitry 720 may be generally arranged toexecute or implement one or more components, modules or logic 722-a. Insome examples circuitry 720 may be configured as an application specificintegrated circuit (ASIC) and at least some components, modules or logic722-a may be implemented as hardware elements of the ASIC. According tosome examples, circuitry 720 may be configured as a field programmablegate array (FPGA) and at least some components, modules or logic 722-amay be implemented as hardware elements of the FPGA.

In some examples, apparatus 700 may include a receive logic 722-1.Receive logic 722-1 may be executed or implemented by circuitry 720 toreceive, via a link, management data to configure a trained model to beimplemented by an inference module for a deep learning application. Forthese examples, the management data may be included in management data710.

According to some examples, apparatus 700 may also include an identifylogic 722-2. Identify logic 722-2 may be executed or implemented bycircuitry 720 to identify the inference module to implement the trainedmodel. For these examples, identify logic 722-2 may maintain mappinginformation 723-a (e.g., in a lookup table) that may indicate whichtrained models are being implemented by which inference module fromamong a plurality of inference modules. Identify logic 722-2 may usemapping information 723-a to identify the inference module.

In some examples, registration information 740 may include informationto be used by identify logic 722-2 to update mapping information 723-a.The information included in information 740 may be sent from inferencemodules and may indicate what new trained models have been received andregistered by these inference modules.

In some examples, apparatus 700 may also include a command logic 722-3.Command logic 722-3 may be executed or implemented by circuitry 720 tocause the inference module to be configured responsive to the managementdata based on one or more commands routed to the inference module via anOOB link. For these examples, the trained model may be implemented bythe inference module using input data received via an inband link. Theone or more commands may be included in commands 730.

Similar to what was mentioned previously, various components, modules orlogic of apparatus 700 may be communicatively coupled to each other byvarious types of communications media to coordinate operations.

FIG. 8 illustrates an example logic flow 800. Logic flow 800 may berepresentative of some or all of the operations executed by one or morelogic or features described herein, such as logic included in apparatus700. More particularly, logic flow 800 may be implemented by at leastreceive logic 722-1, identify logic 722-2 or command logic 722-3.

According to some examples, logic flow 800 at block 802 may receivemanagement data to configure a trained model to be implemented by aninference module for a deep learning application. For these examples,receive logic 722-1 may receive the management data.

In some examples, logic flow 800 at block 804 may identify the inferencemodule to implement the trained model. For these examples, identifylogic 722-2 may identify the inference module.

According to some examples, logic flow 800 at block 806 may cause thetrained model to be configured responsive to the management data basedon one or more commands routed to the inference module via an OOB link,the trained model implemented by the inference module using input datareceived via an inband link. For these examples, command logic 722-3cause the inference module to be configured responsive to the managementdata based on the one or more commands.

FIG. 9 illustrates an example storage medium 900. As shown in FIG. 9,the first storage medium includes a storage medium 900. The storagemedium 900 may comprise an article of manufacture. In some examples,storage medium 900 may include any non-transitory computer readablemedium or machine readable medium, such as an optical, magnetic orsemiconductor storage. Storage medium 900 may store various types ofcomputer executable instructions, such as instructions to implementlogic flow 800. Examples of a computer readable or machine readablestorage medium may include any tangible media capable of storingelectronic data, including volatile memory or non-volatile memory,removable or non-removable memory, erasable or non-erasable memory,writeable or re-writeable memory, and so forth. Examples of computerexecutable instructions may include any suitable type of code, such assource code, compiled code, interpreted code, executable code, staticcode, dynamic code, object-oriented code, visual code, and the like. Theexamples are not limited in this context.

FIG. 10 illustrates an example computing system 1000. In some examples,as shown in FIG. 10, computing system 1000 may include trainingresources 1010, inference resources 1020, processing resources 1040,other system resources 1050 or a communications resources 1060.

In some examples, training resources 1010 may include circuitry toexecute or implement logic for apparatus 400 and/or instructionsincluded in storage medium 600. Training resources 1010 may be housedwithin computing system 1000 and may be arranged to operate in a similarmanner to drawer 120 shown in FIG. 1 or drawer 220 shown in FIG. 2.Although not shown in FIG. 10, training resources 1010 may be coupledwith inference resources 1020 via a switch similar to AI training tointerference switch 105/205 shown in FIGS. 1/2. Training resources 1010may include various hardware elements, software elements, or acombination of both. Examples of hardware elements may include logicdevices, components, processor circuits, circuit elements (e.g.,transistors, resistors, capacitors, inductors, and so forth), integratedcircuits, programmable logic devices (PLDs), digital signal processors(DSPs), memory units, logic gates, registers and so forth. Examples ofsoftware elements may include instruction sets, computing code, computercode, code segments, computer code segments, words, values, symbols, orany combination thereof.

According to some examples, inference resources 1020 may includecircuitry to execute or implement logic for apparatus 700 and/orinstructions included in storage medium 900. Inference resources 1020may also be housed within computing system 1000 and may be arranged tooperate in a similar manner to drawer 120 shown in FIG. 1 or drawer 230shown in FIG. 2. Inference resources 1020 may include various hardwareelements, software elements, or a combination of both. Examples ofhardware and software elements may be similar to but are not limited tothe hardware and software elements mentioned above for trainingresources 1010.

According to some examples, processing resource 1040 may executeprocessing operations or logic for elements of computing system 1000such as an operating system and one or more applications (not shown).Processing resources 1040 may include various hardware elements,software elements, or a combination of both. Examples of hardwareelements may include devices, logic devices, components, processors,microprocessors, circuits, processor circuits, circuit elements,integrated circuits, ASIC, PLDs, DSPs, FPGA, memory units, storageunits, logic gates, registers, semiconductor device, chips, microchips,chip sets, and so forth. Examples of software elements may includesoftware components, programs, applications, computer programs,application programs, device drivers, system programs, softwaredevelopment programs, machine programs, operating system software,middleware, firmware, software modules, routines, subroutines,functions, methods, procedures, software interfaces, application programinterfaces (API), instruction sets, computing code, computer code, codesegments, computer code segments, words, values, symbols, or anycombination thereof.

According to some examples, for training resources 1010, inferenceresources 1020 or processing resources 1040, determining whether anexample is implemented using hardware elements and/or software elementsmay vary in accordance with any number of factors. The any number offactors may include, but are not limited to, desired computational rate,power levels, heat tolerances, processing cycle budget, input datarates, output data rates, memory resources, data bus speeds and otherdesign or performance constraints, as desired for a given example.

In some examples, other system resources 1050 may include commoncomputing elements, memory units, chipsets, controllers, peripherals,interfaces, oscillators, timing devices, video cards, audio cards,multimedia input/output (I/O) components (e.g., digital displays), powersupplies, and so forth. Examples of memory units or memory devices mayinclude without limitation various types of computer readable andmachine readable storage media in the form of one or more higher speedmemory units, such as read-only memory (ROM), RAM, DRAM, DDR SRAM),SRAM, programmable ROM (PROM), erasable programmable ROM (EPROM),electrically erasable programmable ROM (EEPROM), flash memory,multi-threshold level NAND flash memory, polymer memory such asferroelectric polymer memory, ovonic memory, phase change orferroelectric memory, 3-D cross-point memory, FeTRAM, MRAM, STT-MRAM,silicon-oxide-nitride-oxide-silicon (SONOS) memory, magnetic or opticalcards, an array of devices such as Redundant Array of Independent Disks(RAID) drives, solid state memory devices (e.g., USB memory), solidstate drives (SSD) and any other type of storage media suitable forstoring information.

In some examples, communications resources 1060 may include logic and/orfeatures to support one or more communication interfaces coupled withlinks (e.g., to receive management data, training data or input data).For these examples, communications resources 1060 may include one ormore communication interfaces that operate according to variouscommunication protocols or standards to communicate over direct ornetwork communication links. Direct communications may occur via use ofcommunication protocols or standards described in one or more industrystandards (including progenies and variants) such as those associatedwith the PCIe specification. Network communications may occur via use ofcommunication protocols or standards such those described in one or moreEthernet standards promulgated by the Institute of Electrical andElectronics Engineers (IEEE). For example, one such Ethernet standardpromulgated by IEEE may include IEEE 802.3-2012, Carrier sense Multipleaccess with Collision Detection (CSMA/CD) Access Method and PhysicalLayer Specifications, Published in December 2012 (hereinafter “IEEE802.3 specification”). Network communication may also occur according toone or more OpenFlow specifications such as the OpenFlow HardwareAbstraction API Specification. Network communications may also occuraccording to Infiniband Architecture specification.

The components and features of computing system 1000 may be implementedusing any combination of discrete circuitry, ASICs, logic gates and/orsingle chip architectures. Further, the features of computing system1000 may be implemented using microcontrollers, programmable logicarrays and/or microprocessors or any combination of the foregoing wheresuitably appropriate. It is noted that hardware, firmware and/orsoftware elements may be collectively or individually referred to hereinas “logic” or “circuit.”

It should be appreciated that the example computing system 1000 shown inthe block diagram of FIG. 10 may represent one functionally descriptiveexample of many potential implementations. Accordingly, division,omission or inclusion of block functions depicted in the accompanyingfigures does not infer that the hardware components, circuits, softwareand/or elements for implementing these functions would necessarily bedivided, omitted, or included in embodiments.

One or more aspects of at least one example may be implemented byrepresentative instructions stored on at least one machine-readablemedium which represents various logic within the processor, which whenread by a machine, computing device or system causes the machine,computing device or system to fabricate logic to perform the techniquesdescribed herein. Such representations, known as “IP cores” may bestored on a tangible, machine readable medium and supplied to variouscustomers or manufacturing facilities to load into the fabricationmachines that actually make the logic or processor.

Various examples may be implemented using hardware elements, softwareelements, or a combination of both. In some examples, hardware elementsmay include devices, components, processors, microprocessors, circuits,circuit elements (e.g., transistors, resistors, capacitors, inductors,and so forth), integrated circuits, ASICs, PLDs, DSPs, FPGAs, memoryunits, logic gates, registers, semiconductor device, chips, microchips,chip sets, and so forth. In some examples, software elements may includesoftware components, programs, applications, computer programs,application programs, system programs, machine programs, operatingsystem software, middleware, firmware, software modules, routines,subroutines, functions, methods, procedures, software interfaces, APIs,instruction sets, computing code, computer code, code segments, computercode segments, words, values, symbols, or any combination thereof.Determining whether an example is implemented using hardware elementsand/or software elements may vary in accordance with any number offactors, such as desired computational rate, power levels, heattolerances, processing cycle budget, input data rates, output datarates, memory resources, data bus speeds and other design or performanceconstraints, as desired for a given implementation.

Some examples may include an article of manufacture or at least onecomputer-readable medium. A computer-readable medium may include anon-transitory storage medium to store logic. In some examples, thenon-transitory storage medium may include one or more types ofcomputer-readable storage media capable of storing electronic data,including volatile memory or non-volatile memory, removable ornon-removable memory, erasable or non-erasable memory, writeable orre-writeable memory, and so forth. In some examples, the logic mayinclude various software elements, such as software components,programs, applications, computer programs, application programs, systemprograms, machine programs, operating system software, middleware,firmware, software modules, routines, subroutines, functions, methods,procedures, software interfaces, API, instruction sets, computing code,computer code, code segments, computer code segments, words, values,symbols, or any combination thereof.

According to some examples, a computer-readable medium may include anon-transitory storage medium to store or maintain instructions thatwhen executed by a machine, computing device or system, cause themachine, computing device or system to perform methods and/or operationsin accordance with the described examples. The instructions may includeany suitable type of code, such as source code, compiled code,interpreted code, executable code, static code, dynamic code, and thelike. The instructions may be implemented according to a predefinedcomputer language, manner or syntax, for instructing a machine,computing device or system to perform a certain function. Theinstructions may be implemented using any suitable high-level,low-level, object-oriented, visual, compiled and/or interpretedprogramming language.

Some examples may be described using the expression “in one example” or“an example” along with their derivatives. These terms mean that aparticular feature, structure, or characteristic described in connectionwith the example is included in at least one example. The appearances ofthe phrase “in one example” in various places in the specification arenot necessarily all referring to the same example.

Some examples may be described using the expression “coupled” and“connected” along with their derivatives. These terms are notnecessarily intended as synonyms for each other. For example,descriptions using the terms “connected” and/or “coupled” may indicatethat two or more elements are in direct physical or electrical contactwith each other. The term “coupled” or “coupled with”, however, may alsomean that two or more elements are not in direct contact with eachother, but yet still co-operate or interact with each other.

The follow examples pertain to additional examples of technologiesdisclosed herein.

EXAMPLE 1

An example system may include a plurality of training modules toseparately implement at least one training model for one or morerespective deep learning applications The system may also includetraining management logic coupled with the plurality of training modulesvia respective OOBlinks. The training management logic may receivemanagement data and configure the separately implemented at least onetraining model through the respective OOB links responsive to receivedmanagement data. The system may also include an inband training datainterface and switch coupled with the plurality of training modules viarespective inband links to receive training data destined for a trainingmodel to be implemented by a training module from among the plurality oftraining modules and cause received training data to be routed to thetraining module via an inband link from among the respective inbandlinks.

EXAMPLE 2

The system of example 1, the plurality of training modules may include aplurality of neural network processors.

EXAMPLE 3

The system of example 1 may also include the training module to generatea trained mode based on the training module using the received trainingdata to implement the training model. The training module may then sendthe trained model to an inference module through a switch coupled withthe training module and the inference module The inference module mayimplement the trained model using input data to generate one or moreresults for the deep learning application.

EXAMPLE 4

The system of example 3, the inference module may be an FPGA and priorto the trained model being sent to the FPGA, the trained model may betranslated to bit-ware to enable the FPGA to be programmed to implementthe trained model.

EXAMPLE 5

The system of example 3, the plurality of training modules and theinference module may be resident on a same platform included in a drawerconfigured to be inserted in a rack.

EXAMPLE 6

The system of example 5, the switch may be resident on the sameplatform.

EXAMPLE 7

The system of example 3, the plurality of training modules may beresident on a first platform included in a first drawer configured to beinserted in a rack and the inference module may be resident on a secondplatform included in a second drawer configured to be inserted in therack.

EXAMPLE 8

The system of example 7, the may be switch resident on a third platformincluded in a third drawer configured to be inserted in the rack.

EXAMPLE 9

The system of example 1, the one or more respective deep learningapplications may include facial or voice recognition for an individualor a group of people.

EXAMPLE 10

An example apparatus may include an interface coupled with a link andcircuity to execute or implement logic. The logic may receive, via thelink, management data to configure a training model to be implemented bya training module for a deep learning application. The logic may alsoidentify the training module to implement the training model and causethe training module to be configured responsive to the management databased on one or more commands routed to the training module via an OOBlink. The training module may receive training data for the trainingmodel via an inband link and generate a trained model based on using thetraining data to implement the training model.

EXAMPLE 11

The apparatus of example 10, the one or more commands may include atleast one of start implementation of the training model, stopimplementation of the training model, or change one or more prioritiesassociated with implementing the training model.

EXAMPLE 12

The apparatus of example 10, the trained model may be implemented by aninference module coupled with the training module through a switch.

EXAMPLE 13

The apparatus of example 12, the training module may be a neural networkprocessor and the inference module may be an FPGA.

EXAMPLE 14

The apparatus of example 10, the deep learning application may be facialor voice recognition for an individual or a group of people.

EXAMPLE 15

An example method may include receiving management data to configure atraining model to be implemented by a training module for a deeplearning application. The method may also include identifying thetraining module to implement the training model. The method may alsoinclude configuring, responsive to the management data, the trainingmodel for implementation by the training module based on one or morecommands routed to the training module via an OOB link. The method mayalso include receiving training data for the training model and routingthe training data to the training module via an inband link. The methodmay also include generating a trained model based on the training moduleusing the training data to implement the training model. The method mayalso include sending the trained model to an inference module through aswitch coupled with the training module and the inference module, theinference module to implement the trained model using input data togenerate one or more results for the deep learning application.

EXAMPLE 16

The method of example 15, the inference module may include an FPGA. Thetrained model may be translated to bit-ware prior to sending the trainedmodel to enable the FPGA to be programmed to implement the trainedmodel.

EXAMPLE 17

The method of example 15, the training module may include a neuralnetwork processor.

EXAMPLE 18

The method of example 15, identifying the training module may includeidentifying the training module from among a plurality of trainingmodules resident on a same platform included in a drawer configured tobe inserted in a rack.

EXAMPLE 19

The method of example 15, the training module and the inference modulemay be resident on a same platform included in a drawer configured to beinserted in a rack.

EXAMPLE 20

The method of example 19, the switch may be coupled with the trainingmodule and the inference module is resident on the same platform.

EXAMPLE 21

The method of example 15, the training module may be resident on a firstplatform included in a first drawer configured to be inserted in a rackand the inference module resident on a second platform included in asecond drawer configured to be inserted in the rack.

EXAMPLE 22

The method of example 21, the switch may be coupled with the trainingmodule and the inference module may be resident on a third platformincluded in a third drawer configured to be inserted in the rack.

EXAMPLE 23

The method of example 15, the deep learning application may be facial orvoice recognition for an individual or a group of people.

EXAMPLE 24

An example at least one machine readable medium may include a pluralityof instructions that in response to being executed by a system may causethe system to carry out a method according to any one of examples 15 to23.

EXAMPLE 25

An example apparatus may include means for performing the methods of anyone of examples 15 to 23.

EXAMPLE 26

An example system may include a plurality of inference modules toseparately implement at least one trained model for one or morerespective deep learning applications. The system may also includetrained management logic coupled with the plurality of inference modulesvia respective OOB links. The trained management logic may receivemanagement data and configure the separately implemented at least onetrained model through the respective OOB links responsive to receivedmanagement data. The system may also include an inband input datainterface and switch coupled with the plurality of inference modules viarespective inband links to receive input data destined for a trainedmodel to be implemented by an inference module from among the pluralityof inference modules and route the input data to the inference modulevia an inband link from among the respective inband links.

EXAMPLE 27

The system of example 26, the plurality of inference modules may includea plurality of FPGAs.

EXAMPLE 28

The system of example 27, the trained model to implemented by theinference module may to be implemented by an FPGA from among theplurality of FPGAs, the trained model generated by a training modulecoupled with the FPGA through a switch, the trained model may betranslated by the training module to bit-ware prior to sending thetrained model to enable the FPGA to be programmed to implement thetrained model.

EXAMPLE 29

The system of example 28, the training module and the plurality of FPGAsmay be resident on a same platform included in a drawer configured to beinserted in a rack.

EXAMPLE 30

The system of example 29, the switch may be resident on the sameplatform.

EXAMPLE 31

The system of example 29, the plurality of FPGAs may be resident on afirst platform included in a first drawer configured to be inserted in arack and the training module may be resident on a second platformincluded in a second drawer configured to be inserted in the rack.

EXAMPLE 32

The system of example 31, the switch may be resident on a third platformincluded in a third drawer configured to be inserted in the rack.

EXAMPLE 33

The system of example 26, the one or more respective deep learningapplications may include facial or voice recognition for an individualor a group of people.

EXAMPLE 34

An example apparatus may include an interface coupled with a link. Theapparatus may also include circuity to execute or implement logic. thelogic may receive, via the link, management data to configure a trainedmodel to be implemented by an inference module for a deep learningapplication. The logic may also identify the inference module toimplement the trained model and cause the trained model to be configuredresponsive to the management data based on one or more commands routedto the inference module via an OOB link. The trained model may beimplemented by the inference module using input data received via aninband link.

EXAMPLE 35

The apparatus of example 34, the one or more commands may include atleast one of start implementation of the trained model, stopimplementation of the trained model, or change one or more prioritiesassociated with implementing the trained model.

EXAMPLE 36

The apparatus of example 34, the trained model may be received from atraining module coupled with the inference module through a switch.

EXAMPLE 37

The apparatus of example 34, the training module may be a neural networkprocessor and the inference module is a FPGA.

EXAMPLE 38

The apparatus of example 34, the deep learning application may be facialor voice recognition for an individual or a group of people.

EXAMPLE 39

An example method may include receiving management data to configure atrained model to be implemented by an inference module for a deeplearning application. The method may also include identifying theinference module to implement the trained model. The method may alsoinclude configuring the trained model for implementation by theinference module based on one or more commands routed to the inferencemodule via an OOB link. The method may also include receiving input datafor the trained model and routing the input data to the trained modelvia an inband link. The method may also include implementing the trainedmodel at the inference module using the received input data to generateone or more results for the deep learning application.

EXAMPLE 40

The method of example 39, identifying the inference module may includeidentifying the inference module from among a plurality of inferencemodules resident on a same platform included in a drawer configured tobe inserted in a rack.

EXAMPLE 41

The method of example 39, the inference module may include an FPGA.

EXAMPLE 42

The method of example 41, the trained model may be generated by atraining module coupled with the FPGA through a switch. the trainedmodel may be translated by the training module to bit-ware prior tosending the trained model to enable the FPGA to be programmed toimplement the trained model.

EXAMPLE 43

The method of example 42, the training module and the FPGA may beresident on a same platform included in a drawer configured to beinserted in a rack.

EXAMPLE 44

The method of example 43, the switch may be resident on the sameplatform.

EXAMPLE 45

The method of example 42, the FPGA may be resident on a first platformincluded in a first drawer configured to be inserted in a rack and thetraining module resident on a second platform included in a seconddrawer configured to be inserted in the rack.

EXAMPLE 46

The method of example 45, the switch may be resident on a third platformincluded in a third drawer configured to be inserted in the rack.

EXAMPLE 47

The method of example 41, the deep learning application may be facial orvoice recognition for an individual or a group of people.

EXAMPLE 48

An example at least one machine readable medium may include a pluralityof instructions that in response to being executed by a system may causethe system to carry out a method according to any one of examples 39 to47.

EXAMPLE 49

An example apparatus may include means for performing the methods of anyone of examples 39 to 47.

It is emphasized that the Abstract of the Disclosure is provided tocomply with 37 C.F.R. Section 1.72(b), requiring an abstract that willallow the reader to quickly ascertain the nature of the technicaldisclosure. It is submitted with the understanding that it will not beused to interpret or limit the scope or meaning of the claims. Inaddition, in the foregoing Detailed Description, it can be seen thatvarious features are grouped together in a single example for thepurpose of streamlining the disclosure. This method of disclosure is notto be interpreted as reflecting an intention that the claimed examplesrequire more features than are expressly recited in each claim. Rather,as the following claims reflect, inventive subject matter lies in lessthan all features of a single disclosed example. Thus, the followingclaims are hereby incorporated into the Detailed Description, with eachclaim standing on its own as a separate example. In the appended claims,the terms “including” and “in which” are used as the plain-Englishequivalents of the respective terms “comprising” and “wherein,”respectively. Moreover, the terms “first,” “second,” “third,” and soforth, are used merely as labels, and are not intended to imposenumerical requirements on their objects.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

1-9. (canceled)
 10. Computing system configurable for use in artificial intelligence related operations, the computing system comprising: training resource circuitry comprising graphics processing unit (GPU) circuitry, the training resource circuitry being configurable for use in one or more training-related operations; inference resource circuitry comprising central processing unit (CPU) circuitry, the inference resource circuitry being configurable for use in one or more inference-related operations; switch chip circuitry for use, when the computing system is in operation, in direct GPU to GPU communication, via respective communication links, associated with the GPU circuitry; and network interface and switch circuitry to receive, when the computing system is in the operation, training-related data and inference request data, the training-related data being for use in configuring the training resource circuitry to implement, via the one or more training-related operations, one or more training models, the one or more inference-related operations are to be implemented based upon the inference request data; wherein: the one more training models are to be configured, based upon input training data, via the one or more training operations, to generate one or more trained models; the one or more inference-related operations are to implement the one or more trained models; the inference request data is to request that the one or more trained models be implemented via the one or more inference-related operations; the training resource circuitry and the inference resource circuitry comprise certain components; different respective sets of data communication links are to be used in data transmissions involving the certain components; the different respective sets of data communication links comprise one or more sets of data communication links and one or more other sets of data communication links; the one or more sets of data communication links supports one or more communication bandwidths that are one or more relatively higher bandwidths compared to one or more other relatively lower bandwidths supported by the one or more other sets of data communication links; the different respective sets of data communication links are to be used in data communication between or among respective subsets of the certain components; and the computing system is configured for insertion in and coupling to another computing system via a backplane and/or rack.
 11. The computing system of claim 10, wherein: the switch chip circuitry is optimized for the direct GPU to GPU communication.
 12. The computing system of claim 10, wherein: the artificial intelligence-related operations are at least partially related to: facial recognition; voice recognition; and/or image recognition.
 13. The computing system of claim 10, wherein: the GPU circuitry comprises graphics processing units; the CPU circuitry comprises central processing units; the computing system comprises one or more power supplies; and the computing system comprises PCIe communication resources.
 14. Computing system configurable for use in artificial intelligence-related operations, the computing system comprising: training resource circuitry configurable for use in one or more training-related operations, the training resource circuitry comprising artificial intelligence accelerator circuitry, the artificial intelligence accelerator circuitry comprising neural network processing accelerator circuitry and field programmable gate array circuitry; inference resource circuitry configurable for use in one or more inference-related operations, the inference-related resource circuitry comprising central processing unit (CPU) circuitry; direct accelerator-to-accelerator communication circuitry for use in direct accelerator to accelerator communication via respective accelerator-related links; and network communication resource circuitry for use in receiving network-related data; wherein: the training resource circuitry and the inference resource circuitry comprise certain components; different respective sets of data communication links are to be used in data transmissions involving the certain components; the different respective sets of data communication links comprise one or more sets of data communication links and one or more other sets of data communication links; the one or more sets of data communication links supports one or more communication bandwidths that are one or more relatively higher bandwidths compared to one or more other relatively lower bandwidths supported by the one or more other sets of data communication links; the different respective sets of data communication links are to be used in data communication between or among respective subsets of the certain components; and the computing system is configured to be coupled via a backplane and/or rack.
 15. The computing system of claim 14, wherein: the direct accelerator-to-accelerator communication circuitry is optimized for the direct accelerator to accelerator communication.
 16. The computing system of claim 14, wherein: the artificial intelligence-related operations are at least partially related to: at least one security-related application; facial recognition; voice recognition; and/or image recognition.
 17. The computing system of claim 14, wherein: the one or training-related operations and/or the one or more inference-related operations are to be implemented in accordance with one or more service agreements.
 18. The computing system of claim 14, wherein: the CPU circuitry comprises multiple central processing units; the computing system includes one or more power supplies; the computing system comprises PCIe communication resources; and/or the one or more inference-related operations are to implement, at least in part, one or more trained models. 